1. Field of the Invention
The present invention relates to prevention of registration error of a real-space image and a virtual-space image, which is caused when combining the virtual-space image with the real-space image.
2. Description of the Related Art
Recently, studies on mixed reality (hereinafter abbreviated as “MR”) carried out to achieve seamless mixture of a real space and a virtual space are more active than ever.
The aim of MR is to achieve coexistence of a virtual-reality (hereinafter abbreviated as “VR”) world, which is conventionally capable of being experienced in circumstances separated from the real space, and the real space. MR has attracted attention as a technique for improving VR.
Expected applications of MR include new areas quite different in quality from those of conventional VR, such as medical assisting applications, in which a view of the inside of a body is presented to a physician, and work assisting applications in which an assembly procedure for a product overlaid on an actual article in a factory.
A common requirement for these application is a technique for eliminating a “registration error” of the real space and the virtual space. One registration error is a dynamic registration error caused by a time lag produced between a displacement of an MR presenting apparatus and reflection of that displacement in the display of the virtual-space image. Elimination of this registration error (hereinafter referred to as a “dynamic registration error”) is a problem to be solved by the present invention. This will be described in detail.
The MR presenting apparatus measures the position and orientation of a viewpoint for observation of the real space, generates virtual-space image data on the basis of the measurement result, merging a virtual-space image formed by visualizing the generated data and a real-space scene, and displays the merged image. In this method, known as “optical see-through”, the merging is accomplished by means of an optical system such as a half mirror. FIG. 1 illustrates this configuration. It takes time to go through the process from measuring of the position and orientation of the viewpoint to displaying the virtual-space image. When the viewpoint of the observer moves, therefore, the position and orientation of the viewpoint upon looking at the real-space scene would differ from the position and orientation of the observer's viewpoint measured upon generating data for the virtual-space image to be merged with the scene. Merging the real-space scene and the virtual-space image, which have viewpoints with different positions and orientations from each other, produces a registration error of the two images. This is the “dynamic registration error”. FIG. 2 illustrates the occurrence of such a registration error.
Efforts have been made to eliminate the registration error described above.
For example, there is proposed a method for eliminating the “dynamic registration error” of a real-space and a virtual space by moving the display area of a virtual space image in line with movement of the apparatus. The paper “—ReflexHMD—Development of HMD having Vestibule Reflecting Function” (Ryugo KIJIMA, et al., TVRSJ, Vol. 6, No. 2, pp. 107-114, 2001) discloses a method of preparing in-advance virtual-space image data for an area larger than the HMD field of view, cutting an area corresponding to a real-space image based on measured data of the rotation angular velocity data of the HMD as measured by a gyro-sensor from the thus prepared data, displaying the cut area, and merging the same with the real-space scene. This method is effective for eliminating a “dynamic registration error”.
However, the above-mentioned method causes a problem in that it requires a lot of time for processing because of the necessity to generate virtual space image data for an area larger than the field of view. For merging a real space and a virtual space, a process known as “video see-through” is known. This process includes electrically merging, in a merging apparatus such as a computer, a virtual-space image with a real-space image captured in real-time by means of a camera or the like. The above-mentioned method is not applicable to this process, however.